Object recognition device and object recognition system

ABSTRACT

An object recognition device includes a recognition unit, a recognition reliability calculation unit, and a combining unit. A recognition unit recognizes an object by a plurality of functions, based on information which is obtained by measuring an object by a plurality of measurement devices. A recognition reliability calculation unit calculates recognition reliability of recognition results that are obtained by recognizing an object by a recognition unit, for each function. A combining unit combines recognition reliability of the object and recognition results, detects a specified object, and outputs detection results of the specified object.

TECHNICAL FIELD

The present invention relates to, for example, an object recognitiondevice and an object recognition system that recognize a specifiedobject, based on information measured by a measurement device.

BACKGROUND ART

Recently, there is an increasing need for an object recognitiontechnology for detecting an object, based on information measured by ameasurement device. The measurement device is mounted on, for example, acar and is used for automatically recognizing an object (pedestrian orthe like) existing in a travel direction of a car.

A monitoring camera, a distance sensor, a laser radar, an infrared tag,and the like are frequently used as a measurement device. Then, ifintroduction cost for newly installing the measurement device is notrequired and an existing measurement device can be used, it is possibleto introduce an object recognition technology with lower cost. For thisreason, development of a technology is underway in which an existingmonitoring camera is used as a measurement device and an object isdetected based on an image acquired from the monitoring camera. Then, asa technology for recognizing an object from an image, for example, amethod of searching whether or not a portion similar to sample dataexists in an image in a state where lots of sample data (sample images)of an object to be recognized is stored in a database in advance isgenerally used.

However, in a case where appearance of the object differs greatly fromthe sample data due to illumination conditions or the like, it isdifficult to recognize an object by this method. Therefore, attention ispaid to a technology for recognizing an object with high accuracy bycapturing an image of an object using a plurality of cameras andmeasuring a three-dimensional shape of the object. For example, anobject recognition technology which uses a stereo camera is used as sucha technology. In this technology, a distance in a real space from thestereo camera to the object is calculated for each pixel of the image byusing a parallax calculated by comparing a pair of left and right cameraimages captured by the stereo camera. Then, the object is recognizedfrom the image by using three-dimensional information of the objectmeasured based on the calculated distance information.

For example, PTL 1 is cited as an object recognition technology whichuses a stereo camera. PTL 1 discloses that “an obstacle closer to aprescribed distance is recognized based on parallax information of thestereo camera and an obstacle farther than the prescribed distance isrecognized by using pattern recognition”.

CITATION LIST Patent Literature

PTL 1: JP-A-2013-161187

SUMMARY OF INVENTION Technical Problem

The technology disclosed in PTL 1 is an effective technology for a casewhere an in-vehicle stereo camera is used, an object is separated fromthe camera, and an image of the object is captured in right front of thecamera. However, in a case where the stereo camera is installed incommercial facilities, a building, or the like, and an installationposition and an installation angle of the camera differ from place toplace. There is a high possibility that an object approaches the cameraor a shape of the object changes greatly, depending on an installationenvironment of the camera. If a distance between the object and thecamera is too short, the technology disclosed in PTL 1 cannot calculatethe parallaxes at all, and thereby, it is impossible to recognize theobject, based on the three-dimensional information, or to detect theobject even by using a pattern matching due to the fact that a shape ofthe object changes.

The present invention is made in view of such situations, and aims todetect a specified object with high accuracy regardless of aninstallation environment of a measurement device or a position of theobject.

Solution to Problem

An object recognition device according to the present invention includesa recognition unit, a recognition reliability calculation unit, and acombining unit.

A recognition unit recognizes an object by a plurality of methods, basedon information which is obtained by measuring an object by a measurementdevice. A recognition reliability calculation unit calculatesrecognition reliability of recognition results that are obtained byrecognizing an object by a recognition unit, for each method. Acombining unit combines recognition reliability of an object andrecognition results, detects a specified object, and outputs detectionresults of a specified object.

Advantageous Effects of Invention

According to the present invention, it is possible to detect a specifiedobject with high accuracy by combining recognition reliability andrecognition result of an object when the object is recognized by aplurality of methods.

Problems, configurations, and effects other than those described abovewill be clarified by the following description of the embodiments.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an internal configuration exampleof an object recognition system according to a first embodiment of thepresent invention.

FIG. 2 is a hardware configuration diagram of a calculator according tothe first embodiment of the present invention.

FIG. 3 is a flowchart illustrating an example of processing in which acaptured image recognition unit according to the first embodiment of thepresent invention detects a person from a captured image.

FIGS. 4A to 4D are explanatory diagrams illustrating an example ofprocessing in which a three-dimensional recognition unit according tothe first embodiment of the present invention detects a person from aparallax image.

FIG. 5 is a block diagram illustrating an internal configuration exampleof a recognition reliability calculation unit according to the firstembodiment of the present invention.

FIG. 6 is an explanatory diagram illustrating an example of a capturedimage and a parallax image acquired by an image information acquisitionunit according to the first embodiment of the present invention.

FIG. 7 is an explanatory diagram illustrating an example of camerainformation acquired by a camera information acquisition unit accordingto the first embodiment of the present invention, from a database.

FIG. 8 is an explanatory diagram illustrating a first example of camerainformation, an image-capturing range divided into three areas, andrecognition reliability matrix data of each area according to the firstembodiment of the present invention.

FIG. 9 is an explanatory diagram illustrating a second example of animage-capturing range divided into four areas and the recognitionreliability matrix data of each area, according to the first embodimentof the present invention.

FIG. 10 is an explanatory diagram illustrating an example in which arecognition reliability determination unit according to the firstembodiment of the present invention divides the image-capturing rangeinto a plurality of areas and a GUI is used for finally determining therecognition reliability of each area.

FIG. 11 is an explanatory diagram illustrating a display example of arecognition sum value obtained by combining recognition reliability ofimage recognition and recognition reliability of three-dimensionalrecognition according to the first embodiment of the present invention.

FIG. 12 is block diagram illustrating an internal configuration exampleof an object recognition system according to a second embodiment of thepresent invention.

FIG. 13 is a block diagram illustrating an internal configurationexample of a recognition function adjustment unit according to thesecond embodiment of the present invention.

FIG. 14 is a block diagram illustrating an internal configurationexample of an object recognition system according to a third embodimentof the present invention.

DESCRIPTION OF EMBODIMENTS

Hereinafter, a form example for embodying the present invention will bedescribed. In the present specification and the drawings, configurationelements having substantially the same function or configuration aredenoted by the same symbols or reference numerals, and redundantexplanations are omitted.

[1. First Embodiment]

FIG. 1 is a block diagram illustrating a configuration example of anobject recognition system according to a first embodiment of the presentinvention.

The object recognition system 10 illustrated in FIG. 1 includes twocameras 2, an object recognition device 1, a database 9, a display unit11, and an operation unit 12.

The object recognition device 1 includes an image acquisition unit 3, adistance calculation unit 4, a captured image recognition unit 5, athree-dimensional recognition unit 6, a recognition reliabilitycalculation unit 7, and a recognition result combining unit 8. Theobject recognition device 1 recognizes objects in images captured by twoadjacent cameras 2 (an example of measurement device) used as a stereocamera, and detects a specified object from the objects. The objectrecognition device 1 according to the present embodiment sets a personas a specified object that is a detection target, but a different objectsuch as a vehicle may be set as the detection target.

The camera 2 converts an electric signal obtained by converting visiblelight acquired at a predetermined time period through a charge coupleddevice (CCD) imager or a complementary metal oxide semiconductor (CMOS)element, into a digital signal, and generates digital image data.

The image acquisition unit 3 acquires digital image data from the twocameras 2. In the following description, the digital image data acquiredby the image acquisition unit 3 from the camera 2 is referred to as a“captured image”. The captured image is used as an example ofinformation obtained by measuring an object by the camera 2.

The distance calculation unit 4 calculates a distance in a real space toan object viewed from the camera 2, with respect to each pixel in thecaptured image. For example, the distance calculation unit 4 calculatesa distance from the camera 2 o an object in the captured image, based onthe captured images of the two cameras 2 output from the imageacquisition unit 3 and camera parameters previously estimated by a knowncalibration technology in each camera 2. As a method of calculating thedistance, for example, a general method called stereo matching forcalculating parallax by using a basic matrix obtained from cameraparameters is used. Then, the distance calculation unit 4 outputs aparallax image obtained by including distances, which are calculated foreach object, to the objects viewed from the cameras 2 in captured imagesrespectively acquired from the two cameras 2 as distance information, tothe three-dimensional recognition unit 6 and the recognition reliabilitycalculation unit 7. The parallax image is also used as an example ofinformation obtained by measuring an object by the camera 2.

The captured image recognition unit 5 (an example of a recognition unit)recognizes an object, based on a captured image captured by one camera 2of the two cameras 2. Details of processing performed by the capturedimage recognition unit 5 will be described below with reference to FIG.3. In the following description, an object recognition function by whichthe captured image recognition unit 5 recognizes an object is alsoreferred to as “image recognition”.

The three-dimensional recognition unit 6 (an example of a recognitionunit) recognizes an object, based on the distance information includedin the parallax image. Details of the processing performed by thethree-dimensional recognition unit 6 will be described below withreference to FIG. 4. In the following description, an object recognitionfunction by which the three-dimensional recognition unit 6 recognizes anobject is also referred to as “three-dimensional recognition”.

The recognition reliability calculation unit 7 calculates reliability ofimage recognition and recognition reliability of three-dimensionalrecognition for each object recognition function, based on camerainformation of the camera 2 and an installation environment of thecamera 2. In addition, the recognition reliability calculation unit 7can also calculate recognition reliability for each area, based on thecamera information that a user changes by the operation unit 12, adivision method, and the recognition reliability which are acquired fromthe database 9.

The camera information is information including at least one ofinstallation environment and internal information of the camera 2. Thecamera information may include an image-capturing range and a distancefrom the camera 2 to an object. The recognition reliability is a valueindicating reliability of recognition results that the captured imagerecognition unit 5 and the three-dimensional recognition unit recognizea specified object, based on each object recognition function, and thehigher the recognition reliability is, the easier the recognitionresults are to be reflected in detection results of the specified objectdisplayed on the display unit 11. Details of processing performed by therecognition reliability calculation unit 7 will be described below withreference to FIGS. 5 and 6.

The recognition result combining unit 8 (an example of a combining unit)combines recognition reliability of an object recognition function bywhich the captured image recognition unit 5 and the three-dimensionalrecognition unit 6 recognize an object, recognition results of thecaptured image recognition unit 5, and recognition results of thethree-dimensional recognition unit 6, and detects a specified object.Then, the recognition result combining unit 8 outputs detection resultsof the specified object to the display unit 11.

The database 9 records camera parameters, pattern images, camerainformation peculiar to each camera 2, an image-capturing range divisionmethod, recognition reliability matrix data, and the like read which areoutput from the image acquisition unit 3, and the information isfrequently read from each unit of the object recognition device 1. Thedetection result of the object recognition device 1 are recorded in thedatabase 9.

The display unit 11 displays the detection results of the specifiedobject detected by the recognition result combining unit 8. Details ofthe recognition results displayed on the display unit 11 will bedescribed below with reference to FIGS. 10 and 11. The display unit 11is, for example, a liquid crystal display monitor, and displays resultsof processing through a graphical user interface (GUI).

The operation unit 12 is used by the user to perform predeterminedoperation inputs and instructions to the object recognition device 1.For example, a keyboard, a mouse, or the like is used for the operationunit 12. The camera information, the division method, and therecognition reliability can be changed by an input of the user from theoperation unit 12 through the GUI displayed on the display unit 11. Thecamera information, the division method, and the recognition reliabilitywhich are changed are recorded in the database 9.

<Hardware Configuration Example of Calculator>

Next, a hardware configuration of a calculator C configuring the objectrecognition device 1 will be described.

FIG. 2 is a block diagram illustrating a hardware configuration exampleof the calculator C.

The calculator C is hardware used as a so-called computer. Thecalculator C includes a central processing unit (CPU) C1, a read onlymemory (ROM) C2, a random access memory (RAM) C3, a nonvolatile storageC5, and a network interface C6, which are connected to a bus C4.

The CPU C1 reads a software program code for realizing each functionaccording to the present embodiment from the ROM C2 and executes thesoftware program code. Variables, parameters, and the like which aregenerated during arithmetic processing are temporarily written to theRAM C3.

For example, a hard disk drive (HDD), a solid state drive (SSD), aflexible disk, an optical disk, a magneto-optical disk, a CD-ROM, aCD-R, a magnetic tape, a nonvolatile memory card, and the like are usedas the nonvolatile storage C5. In addition to an operating system (OS)and various parameters, a program for causing the calculator C tofunction is recorded in the nonvolatile storage C5. For example, anetwork interface card (NIC) and the like are used for the networkinterface C6, and various data can be transmitted and received via alocal area network (LAN) to which a terminal is connected, a dedicatedline, and the like.

A function of the object recognition device 1 is realized by thecalculator C, but the function of the object recognition device 1 may berealized by configuring the camera 2 itself using the calculator C.

FIG. 3 illustrates an example of processing in which the captured imagerecognition unit 5 detects a person from a captured image.

First, the captured image recognition unit 5 learns a feature pattern(for example, an outline of a head and a shoulder of a person) in atwo-dimensional captured image such as a color and a shape from sampledata (sample image) of a person read from the database 9, and generatesa pattern identifier (S1). Next, the captured image recognition unit 5comprehensively scans the captured image by a detection window (forexample, a rectangular area smaller than the captured image), andextracts a local feature part from the captured image (S2).

Next, the captured image recognition unit 5 uses the pattern identifiergenerated in step S1 to calculate a matching degree (an example ofrecognition results) between the local feature part extracted from thecaptured image and an image feature of the pattern identifier (S3), andoutputs the calculated matching degree to the recognition resultcombining unit 8. If the matching degree of the image feature calculatedin step S3 is equal to or larger than a specified value, the capturedimage recognition unit 5 determines that a person is captured in animage, and if the matching degree is less than the specified value, thecaptured image recognition unit 5 determines that a person is notcaptured in the image.

The above-described image recognition method performed by the capturedimage recognition unit 5 is based on an application of a generallearning type object recognition technique. In addition to this method,as long as a method can detect a person from a captured image, themethod is not limited in particular.

FIG. 4 illustrates an example of processing in which thethree-dimensional recognition unit 6 detects a person from a parallaximage.

FIG. 4A illustrates an example of a parallax image 22.

The parallax image 22 includes a distance value from the camera 2 toeach object captured in the image for each pixel. For example, persons20 a to 20 c which are detection targets and a building 21 which is notthe detection target are captured in the parallax image 22. Then, thethree-dimensional recognition unit 6 performs processing of recognizingthe persons 20 a to 20 c which are detection targets from the parallaximage 22.

FIG. 4B illustrates an outline of processing performed by thethree-dimensional recognition unit 6.

The three-dimensional recognition unit 6 extracts regions of the persons20 a to 20 c by background subtraction processing. Here, thethree-dimensional recognition unit 6 reads a background parallax image23 previously acquired from the database 9. Only the building 21 existsin the background parallax image 23. Accordingly, by taking a differencebetween the parallax image 22 where a person exists and the backgroundparallax image 23, the three-dimensional recognition unit 6 can create aparallax image 24 including a person region where the person exists.Positions of the persons 20 a to 20 c in the parallax image 24 arespecified by image coordinates including x (a position in the xdirection in the parallax image 24), y (a position in the y direction inthe parallax image 24), and L (a distance of a depth from the camera 2to the object).

FIG. 4C illustrates an example of first viewpoint conversion processing.

The three-dimensional recognition unit 6 acquires three-dimensionalinformation of each person distinguished from the person region byperforming the first viewpoint conversion processing.

FIG. 4C illustrates image coordinates of the parallax image 24,three-dimensional information 25 in which a person estimated from thedistance information is oblique, and an overhead view image 26 obtainedby viewing the three-dimensional information 25 from right aboveviewpoint. The overhead view image 26 is created from thethree-dimensional information 25 by a general method such as aperspective projection which uses camera parameters. A three-dimensionalinformation template t1 for specifying the persons 20 a to 20 c isillustrated in the lower left of the overhead view image 26. Thethree-dimensional information template t1 is read from the database 9 bythe three-dimensional recognition unit 6.

For example, the three-dimensional recognition unit 6 comprehensivelyscans the three-dimensional information template t1 in a directionindicated by an arrow in the overhead view image 26, thereby, specifyingan image matching the three-dimensional information template t1 as thepersons 20 a to 20 c. Then, the three-dimensional recognition unit 6calculates a matching degree (an example of recognition results) betweenthe three-dimensional information template t1 and the three-dimensionalinformation of a person captured in the overhead view image 26, andoutputs the calculated matching degree to the recognition resultcombining unit 8. At this time, if the matching degree is equal to orlarger than the specified value, the three-dimensional recognition unit6 can determine that a person is captured in the overhead view image 26,and if the matching degree is less than the specified value, thethree-dimensional recognition unit 6 can determine that the person isnot captured in the overhead view image 26.

FIG. 4D illustrates an example of second viewpoint conversionprocessing.

The three-dimensional recognition unit 6 does not create the overheadview image 26 rom the three-dimensional information 25 of all objectsillustrated in FIG. 4C, can create three-dimensional information 27 inwhich only the head of a person extracted from the person region isoblique is created as illustrated in FIG. 4D, and can also create theoverhead view image 28 from the three-dimensional information 27.Through such processing, the three-dimensional recognition unit 6 caneasily divide persons from each other as compared with the overhead viewimage 26 generated from the three-dimensional information of allpersons.

Then, in the same manner as the first viewpoint conversion processing,the three-dimensional recognition unit 6 calculates a matching degreebetween the three-dimensional information template t2 of the head readfrom the database 9 and the heads of each person, and outputs thecalculated matching degree to the recognition result combining unit 8.Then, the three-dimensional recognition unit 6 can detect a person withhigh accuracy even in a scene where persons are crowded.

As a method of extracting the three-dimensional information 27 of onlythe head from the person region, there is a method of detectingthree-dimensional information above from a certain height above a planeas the head. Besides, as long as a method can extract a region near thehead, the method is not limited in particular.

In addition to the above-described method, the three-dimensionalrecognition unit 6 can use a method capable of detecting a person fromthree-dimensional information. For example, the three-dimensionalrecognition unit 6 may use a method of detecting a person by using notonly the three-dimensional information but also image features. Inaddition, in the same manner as the captured image recognition unit 5,the three-dimensional recognition unit 6 may use a method of detecting aperson by creating a pattern identifier from the three-dimensionalinformation of a person which is sample data read from the database 9,and calculating a matching degree between the acquired three-dimensionalinformation and the sample data.

FIG. 5 is a block diagram illustrating an internal configuration exampleof the recognition reliability calculation unit 7.

The recognition reliability calculation unit 7 illustrated in FIG. 5calculates recognition reliability of an object recognized by thecaptured image recognition unit 5 and recognition reliability of anobject recognized by the three-dimensional recognition unit 6, for eacharea obtained by dividing an image-capturing range of a captured image,based on camera information read from the database 9.

The recognition reliability calculation unit 7 includes an imageinformation acquisition unit 7 a, a camera information acquisition unit7 b, an area information acquisition unit 7 c, and a recognitionreliability determination unit 7 d.

The image information acquisition unit 7 a acquires a captured imagefrom the image acquisition unit 3, acquires a parallax image from thedistance calculation unit 4, and outputs the captured image and theparallax image to the recognition reliability determination unit 7 d.

The camera information acquisition unit 7 b (an example of a measurementdevice information acquisition unit) acquires camera information (anexample of measurement device information) including an installationangle, a focal length, and the like of the camera 2 from the database 9.Then, the camera information acquisition unit 7 b outputs the acquiredcamera information to the recognition reliability determination unit 7d.

The area information acquisition unit 7 c acquires an image-capturingrange division method corresponding to the camera information andrecognition reliability matrix data, from the database 9 for each area,and outputs the image-capturing range division method and therecognition reliability matrix data to the recognition reliabilitydetermination unit 7 d.

The recognition reliability determination unit 7 d divides theimage-capturing range of the camera 2 into areas with a predeterminedposition and a predetermined size, based on the captured image and theparallax image, the camera information, the image-capturing rangedivision method and the recognition reliability matrix data. Then, therecognition reliability determination unit 7 d determines therecognition reliability of the captured image and the recognitionreliability of the parallax image for each of the divided areas, andoutputs the determined recognition reliability for each area to therecognition result combining unit 8.

FIG. 6 illustrates an example of a captured image 30 and a parallaximage 31 which are acquired by the image information acquisition unit 7a.

Persons 32 a to 32 c as an example of objects, a mirror 33, and a wall34 are captured in the captured image 30. Meanwhile, the persons 32 a to32 c as an example of objects, and the wall 34 are also captured in theparallax image 31, but the mirror 33 is not captured. This is due to adifference in properties between the captured image 30 and the parallaximage 31. In the parallax image 31, if the camera 2 is taken as areference, a difference in luminance value at a boundary between objectslocated far away is reduced. However, in the captured image 30, thedifference in luminance value does not change even at the boundarybetween objects located far away. Accordingly, the mirror 33 and thewall 34 can be distinguished from each other in the captured image 30,but the mirror 33 and the wall 34 cannot be distinguished from eachother in the parallax image 31.

FIG. 7 illustrates an example in which the camera informationacquisition unit 7 b acquires the camera information from the database9.

The camera information is divided into two pieces of internalinformation of the camera 2 and an installation environment of thecamera 2. In a case where two cameras 2 are used as a stereo camera, theinternal information of the camera 2 includes a baseline length betweenthe left and right cameras 2, and the like, in addition to internalparameters such as, a resolution, a focal distance, lens distortion, andskew. The installation environment of the camera 2 includes externalparameters such as an installation angle (a pan angle, a tilt angle, aroll angle) of the camera 2 and an installation position (X position, Yposition, Z position) of the camera 2.

The camera information acquisition unit 7 b may acquire camerainformation other than that illustrated in FIG. 7, as long as the camerainformation affects a change in a shape and appearance of an object inthe image. In addition, the camera information acquisition unit 7 b mayacquire camera information manually instructed by a user using a GUI orthe like, or may acquire camera information estimated by using acalibration technology of a captured image or a parallax image.

FIG. 8 illustrates a first example of the camera information, theimage-capturing range divided into three areas, and the recognitionreliability matrix data of each area.

The same camera information as that illustrated in FIG. 7 is stored inthe camera information 40 acquired by the area information acquisitionunit 7 c. Then, the division method illustrates how to divide theimage-capturing range, corresponding to the camera information 40. It isillustrated by the division method 41 that the image-capturing range isdivided into three areas of first to third areas. Matrix data 42illustrated in a list illustrates reliability of image recognition andreliability of three-dimensional recognition for each of the first tothird areas.

The division method 41 of an image-capturing range and the recognitionreliability matrix data 42 may be changed according to algorithms usedfor image recognition and three-dimensional recognition. For example, inan object recognition function which uses templates of all persons,recognition reliability is close to “0” in an area where an image of thewhole body is not captured and in areas of left and right ends of animage whose shape is largely changed due to lens distortion or the like,but the recognition reliabilities of the center of an image where thewhole body is reflected and a distant part of the image are high.Meanwhile, in the image recognition using the template of a part of thehuman body, a person can be detected in the entire area of theimage-capturing range, but recognition reliability decreases as comparedwith image recognition which uses templates of all persons. Accordingly,depending on what kind of template is used to perform the imagerecognition, the image-capturing range is appropriately divided, and thecontents of the matrix data 42 are changed.

In addition, for the recognition reliability matrix data, a total valueof the recognition reliability of two object recognition functions maynot always be set to “100” like the matrix data 42, and the value maynot be limited to a range of “0” to “100”. For example, in a case wherethe recognition reliability is extremely high, the total value of therecognition reliability of the two object recognition functions may beset to a value such as “200”. Depending on the image-capturing range,one object recognition technology may be used by setting the recognitionreliability of one of the two object recognition technologies to “0”.

FIG. 9 illustrates a second example of an image-capturing range dividedinto four areas and the recognition reliability matrix data of eacharea. In FIG. 9, description of the camera information 40 is omitted.

A division method 43 of the image-capturing range acquired by the areainformation acquisition unit 7 c is to divide the image-capturing rangeinto four areas of first to fourth areas in accordance with a distancefrom the camera 2. The matrix data 44 illustrates recognitionreliability of the image recognition and the three-dimensionalrecognition for each of the first to fourth areas. The division method43 of the image-capturing range and the recognition reliability matrixdata 44 may be changed in accordance with distance information from thecamera 2 which is acquired from the parallax image.

The image-capturing range division method is not limited only to thedivision in the vertical direction as illustrated in FIGS. 8 and 9, anda method of dividing the image-capturing range in the horizontaldirection or in a ring shape may be used. Depending on the camerainformation, the image-capturing range may not necessarily be divided,but the entire image-capturing range may be set as one area.

FIG. 10 is an example in which the recognition reliability determinationunit 7 d divides the image-capturing range into a plurality of areas anda GUI is used for finally determining the recognition reliability ofeach area. All screens 50 a and 50 b illustrated in FIG. 10 aredisplayed on the display unit 11.

The recognition reliability determination unit 7 d associates the camerainformation acquired from the camera information acquisition unit 7 bwith the image-capturing range division method and the recognitionreliability matrix data which are acquired from the area informationacquisition unit 7 c, and displays the results on the display unit 11 asthe entire screen 50 a.

In addition to a captured image 51 a divided by the recognitionreliability determination unit 7 d into first to third areas on thebasis of the division method of the image-capturing range, matrix data52 a, camera information 53 and parallax image 54 are displayed on theentire screen 50 a.

The captured image 51 a is an image acquired by the image informationacquisition unit 7 a and is the same as the captured image 30illustrated in FIG. 6.

The matrix data 52 a is information indicating the recognitionreliability of an object recognition function for each of the dividedareas as illustrated in FIGS. 8 and 9.

The camera information 53 is information acquired by the camerainformation acquisition unit 7 b and has the same content as the camerainformation illustrated in FIG. 7.

The parallax image 54 is an image acquired by the image informationacquisition unit 7 a and is the same as the parallax image 31illustrated in FIG. 6.

A user views entirety of the captured image 51 a and confirms how animage-capturing range of the captured image 51 a is divided by thedivision method 41. Then, the user modifies an image-capturing rangedivision method and the recognition reliability for each area, by theoperation unit 12, based on content of a captured image. The entirescreen 50 b illustrates an example in which the user modifies theimage-capturing range division method and the matrix data 52 a.

Here, as illustrated in the entire screen 50 a, the mirror 33 exists inthe captured image 51 a acquired by the image information acquisitionunit 7 a, in addition to a person. Since the three-dimensionalrecognition unit 6 recognizes an object, based on the parallax image 54in which the mirror 33 and the wall 34 are not distinguished asillustrated in FIG. 6, there is a low possibility of detecting a persondisplayed on the mirror 33. However, since the captured imagerecognition unit 5 recognizes an object, based on the feature of animage in the captured image 51 a, there is a high possibility oferroneously detecting a person displayed on the mirror 33.

Therefore, a user performs an input from the operation unit 12 throughthe GUI, and creates the captured image 51 b obtained by dividing a newarea (fourth area) surrounding the mirror 33 from the captured image 51a. Then, the user modifies the recognition reliability of imagerecognition in the fourth area which is added to the matrix data 52 b to0% and recognition reliability of three-dimensional recognition which isadded to the matrix data 52 b to 90%. Thereby, for an object displayedon the fourth area, recognition results of three-dimensional recognitionhave higher reliability than recognition results of image recognition.

As a method by which a user modifies the recognition reliability of anarea, for example, there is a method of dividing an area of a partexpected to have large variation in lighting conditions, and modifyingsuch that the recognition reliability of image recognition in the areais decreased and the recognition reliability of three-dimensionalrecognition is significantly increased. In addition, there is also amethod of decreasing reliability of the three-dimensional recognition ata part where parallax is not sufficiently obtained by the parallax image54.

In addition, it is also possible for a user to perform an input from theoperation unit 12 via the GUI and to modify a value of the camerainformation 53. If the camera information 53 is modified by the user,the area information acquisition unit 7 c acquires the image-capturingrange division method and recognition reliability matrix data of eacharea, based on the modified camera information 53. Accordingly, contentof the captured images 51 a and 51 b and the matrix data 52 a and 52 bare automatically updated.

In addition, various methods are used to associate the camerainformation acquired by the camera information acquisition unit 7 b withthe image-capturing range division method and the recognitionreliability matrix data which are acquired by the area informationacquisition unit 7 c. For example, there is a method by which the areainformation acquisition unit 7 c uses the database 9 in which theimage-capturing range division method and the recognition reliabilitymatrix data are previously stored for every camera information acquiredby the camera information acquisition unit 7 b. In addition, there isalso a method of using the database 9 in which the image-capturing rangedivision method and the recognition reliability matrix data arepreviously stored for each distance from the camera 2 to an object.Thereby, the area information acquisition unit 7 c can acquire theimage-capturing range division method and the recognition reliabilitymatrix data from the database 9 only by acquiring the camera informationacquired by the camera information acquisition unit 7 b, withoutrequiring association of various types of information performed by therecognition reliability determination unit 7 d.

In addition, the area information acquisition unit 7 c may use thedatabase 9 in which distance information acquired from a parallax imageis associated with the image-capturing range division method and therecognition reliability matrix data. In addition, the area informationacquisition unit 7 c may be the image-capturing range division methodand the recognition reliability matrix data associated with both thecamera information and the distance information of the camera 2. Then, auser may be able to select the image-capturing range division method andthe recognition reliability matrix data corresponding to the camerainformation through the GUI.

In addition, the user may use the distance information through the GUIand change the recognition reliability in accordance with a distance.For example, a user may increase recognition reliabilities of imagerecognition and three-dimensional recognition at a near area, anddecrease the recognition reliabilities of the image recognition and thethree-dimensional recognition at a distant area.

In addition, the image-capturing range division method and imagerecognition reliability matrix data are associated with the imageinformation (the captured image and the parallax image) acquired by theimage information acquisition unit 7 a to be stored in advance in thedatabase 9. The recognition reliability calculation unit 7 collates thecaptured image acquired by the image information acquisition unit 7 ausing a similar image searching technology or the like with the imageinformation stored in the database 9. Then, the recognition reliabilitycalculation unit 7 reads the image-capturing range division method andthe image recognition reliability matrix data which are associated withthe matched image information, from the database 9. Thereby, the areainformation acquisition unit 7 c can acquire the image-capturing rangedivision method and the image recognition reliability matrix data fromthe database 9 without using the camera information.

In addition, when a user divides an image-capturing range into aplurality of areas through the GUI, the recognition reliabilitycalculation unit 7 may search the database 9 for an image-capturingrange division method similar to the image-capturing range divisionmethod performed by a user, based on input information input from theoperation unit 12 and the camera information. Then, the recognitionreliability calculation unit 7 may acquire the most similarimage-capturing range division method and recognition reliability matrixdata corresponding to the division method from the database 9, and maydisplay the recognition reliability matrix data on the display unit 11.

FIG. 11 illustrates a display example of a recognition sum valueobtained by combining recognition reliability of image recognition andrecognition reliability of three-dimensional recognition. The GUIillustrated in FIG. 11 is displayed on the display unit 11.

Actual captured image 60, recognition reliability matrix data 61, and arecognition result list 64 are displayed in the GUI illustrated in FIG.11. The captured image 60 is divided into first to fourth areas by therecognition reliability calculation unit 7. An object 62 a is includedin the first area, an object 62 b is included in the second area, anobject 62 c is included in the third area, and an object 63 is includedin the fourth area.

The recognition reliability of image recognition of the first to fourthareas and the recognition reliability of three-dimensional recognitionare displayed in the matrix data 61. Then, a matching degree betweenobjects recognized by the captured image recognition unit 5 and thethree-dimensional recognition unit 6 in the first to fourth areas, therecognition reliability of image recognition, the recognitionreliability of three-dimensional recognition, and the recognition sumvalue are displayed in the recognition result list 64.

Here, the recognition result combining unit 8 combines a degreeindicating that an object recognized by the captured image recognitionunit 5 from a captured image is a specified object and the recognitionreliability for recognizing an object in the captured image. Inaddition, the recognition result combining unit 8 combines a degreeindicating that an object recognized by the three-dimensionalrecognition unit 66 from the parallax image is a specified object andthe recognition reliability for recognizing an object in the parallaximage. Then, in a case where a recognition sum value calculated bycombining a matching degree of image recognition and a matching degreeof three-dimensional recognition, based on the recognition reliabilityis equal to or larger than a predetermined value, the recognition resultcombining unit 8 sets the specified object to the detection results(indicating presence or absence of a person displayed in the capturedimage 60).

$\begin{matrix}{{{Recognition}\mspace{14mu}{sum}\mspace{14mu}{value}} = {\frac{\begin{matrix}{\left( {{matching}\mspace{14mu}{degree}\mspace{14mu}{of}\mspace{14mu}{image}\mspace{14mu}{recognition}} \right) \times} \\\left( {{recognition}\mspace{14mu}{reliability}\mspace{14mu}{of}\mspace{14mu}{image}\mspace{14mu}{recognition}} \right)\end{matrix}}{100} + \frac{\begin{matrix}{\left( {{matching}\mspace{14mu}{degree}\mspace{14mu}{of}\mspace{14mu}{three}\text{-}{dimensional}\mspace{14mu}{recognition}} \right) \times} \\\left( {{recognition}\mspace{14mu}{reliability}\mspace{14mu}{of}\mspace{14mu}{three}\text{-}{dimensional}\mspace{14mu}{recognition}} \right)\end{matrix}}{100}}} & (1)\end{matrix}$

A matching degree of image recognition or three-dimensional recognitionin the recognition result list 64 is normalized between “0” and “10”. Amethod of normalizing the matching degree is not limited. Therecognition result combining unit 8 finally obtains recognition resultsof a person by determining that, if a value equal to or larger than apreset appropriate threshold value is illustrated, the recognized objectis a person, and if a value less than the threshold value isillustrated, the recognized object is not a person, by using therecognition sum value. For example, by setting the threshold value to“3” among the recognition sum values illustrated in FIG. 11, only theobjects 62 a, 62 b, and 62 c are detected as persons, and the object 63reflected in the mirror is detected to be not a person.

In an algorithm of the image recognition or the three-dimensionalrecognition used by the captured image recognition unit 5 or thethree-dimensional recognition unit 6, there is a case where a matchingdegree cannot be calculated and instead it is possible to acquire onlywhether or not a person exists. In this case, the recognition resultcombining unit 8 may obtain the recognition sum value by setting amatching degree in a case where a person exists to “10” and a matchingdegree in a case where a person does not exist to “0”.

In addition, the recognition result combining unit 8 may select onlyresults of an object recognition function with high recognitionreliability for each area instead of combining results of two objectrecognition functions.

The object recognition device 1 described above recognizes combinesrecognition results of the image recognition and the three-dimensionalrecognition by using the camera information of the two cameras 2, theimage-capturing range division method and the matrix data storing therecognition reliability of each area. Thereby, the object recognitiondevice 1 can detect a specified object with high accuracy regardless ofan installation environment of the camera 2 or a position of an object.

In addition, when a specified object is near the camera 2, the objectrecognition device 1 can detect the specified object from an objectrecognized based on a captured image. In addition, when a specifiedobject is far from the camera 2, the object recognition device 1 candetect the specified object from an object recognized based on aparallax image.

In the first embodiment, an example of a case where a stereo camera isused as a measurement device, and an image recognition in which an imageis used and three-dimensional recognition in which distance informationis used are used as an object recognition method is described, but thefirst example is not limited to the measurement device and the objectrecognition method.

Even if heights of the two cameras 2 and a distance to the objectchange, the recognition reliability calculation unit 7 corrects acaptured image and a parallax image so as to be in the sameimage-capturing range, based on camera information of the two cameras 2read from the database 9. Accordingly, it is possible to easilyconstruct the object recognition system 1 by using the existing cameras2.

In addition to using the two cameras 2 which are measurement devices asadjacent stereo cameras, the object recognition device 1 may acquireimage information of an object and distance information by using threeor more cameras 2, and may perform a series of processing. In this case,since a range of the captured image and the parallax image recognized bythe object recognition device 1 is enlarged, it is possible to detect aspecified object from a wide range.

In addition, in the first embodiment, an example is described in whichtwo object recognition functions are used as an object recognitiontechnology, but the number and the type of object recognition functionsto be used are not limited. For example, in a case where a person isdetected as a specified object, three object recognition functions of afunction of detecting a head by using image recognition, a function ofdetecting a person by using the image recognition, and a function ofperforming three-dimensional recognition by using distance informationmay be used in combination.

[2. Second Embodiment]

Next, an object recognition system according to a second embodiment ofthe present invention will be described.

FIG. 12 illustrates an internal configuration example of an objectrecognition system 10A according to the second embodiment.

The object recognition system 10A includes two cameras 2, an objectrecognition device 1A, a database 9, a display unit 11, and an operationunit 12.

In addition to the image acquisition unit 3, the distance calculationunit 4, the captured image recognition unit 5, the three-dimensionalrecognition unit 6, the recognition reliability calculation unit 7A, andthe recognition result combining unit 8, the object recognition device1A includes a recognition function adjustment unit 70 and a recognitionreliability updating unit 71.

The recognition function adjustment unit 70 adjusts a parameter (afunction of the captured image recognition unit to recognize an object)used when the captured image recognition unit 5 recognizes an object,based on camera information. In addition, the recognition functionadjustment unit 70 adjusts a parameter (an example of a function of thethree-dimensional recognition unit 6 to recognize an object) used whenthe three-dimensional recognition unit 6 recognizes an object, based onthe camera information.

The recognition reliability updating unit 71 updates matrix data inwhich recognition reliability for each divided range of animage-capturing range is stored, based on the parameter adjusted by therecognition function adjustment unit 70. The updated matrix data isrecorded in the database 9. Hereinafter, details of functions of therecognition function adjustment unit 70 and the recognition reliabilityupdating unit 71 will be described.

FIG. 13 illustrates an internal configuration example of the recognitionfunction adjustment unit 70.

Here, a function of adjusting a parameter of an object recognitionfunction by the recognition function adjustment unit 70 will bedescribed.

The recognition function adjustment unit 70 includes a camerainformation acquisition unit 70 a, an area information acquisition unit70 b, an image information acquisition unit 70 c, a pattern identifierreconstruction unit 70 d, and a recognition parameter determination unit70 e.

The camera information acquisition unit 70 a acquires camera informationof the camera 2 from the database 9, and outputs the camera informationto the pattern identifier reconstruction unit 70 d and the recognitionparameter determination unit 70 e.

The area information acquisition unit 70 b acquires an image-capturingrange division method and recognition reliability matrix data of thecamera 2, which are associated with the camera information acquired fromthe database 9. Then, the area information acquisition unit 70 b outputsthe image-capturing range division method and the recognitionreliability matrix data of the camera 2 to the pattern identifierreconstruction unit 70 d and the recognition parameter determinationunit 70 e.

The image information acquisition unit 70 c acquires a captured imagefrom the image acquisition unit 3, acquires a parallax image from thedistance calculation unit 4, and outputs the captured image and theparallax image to the pattern identifier reconstruction unit 70 d andthe recognition parameter determination unit 70 e.

The pattern identifier reconstruction unit 70 d performs predeterminedprocessing by using the captured image and the parallax image acquiredby the image information acquisition unit 70 c, the camera informationacquired by the camera information acquisition unit 70 a, and theimage-capturing range division method and the recognition reliabilitymatrix data acquired by the area information acquisition unit 70 b. Atthis time, the pattern identifier reconstruction unit 70 d reconstructsa pattern identifier used by the captured image recognition unit 5 bychanging sample data (sample image) used by the captured imagerecognition unit 5, and records the pattern identifier in the database9. The captured image recognition unit 5 according to the firstembodiment described above generates the pattern identifier, but thecaptured image recognition unit 5 according to the present embodimentreads the pattern identifier reconstructed by the pattern identifierreconstruction unit 70 d from the database 9, and uses the patternidentifier for processing.

The database 9 records in advance sample data for each installationenvironment of the camera 2 such as a height or an installation angle ofthe camera 2. Then, the pattern identifier reconstruction unit 70 dchanges to sample data of an installation environment most similar tothe camera information (installation environment of the camera 2)acquired by the camera information acquisition unit 70 a. The patternidentifier reconstruction unit 70 d may perform scene recognition andthe like, based on a captured image or a parallax image, and may changethe scene recognition and the like into sample data close to theinstallation environment of the camera 2.

In addition to this, one piece of sample data is recorded in thedatabase 9, and the sample data may be changed by performing distortioncorrection processing and the like with respect to an image in thesample data by the pattern identifier reconstructing unit 70 d, based onthe camera information such as lens distortion or a focal distance ofthe camera 2. In addition, the sample data may be changed by imageprocessing such as changing a size of the image in the sample data orthe like by the pattern identifier reconstructing unit 70 d, based ondistance information acquired from a parallax image.

In addition, the pattern identifier reconstruction unit 70 d mayreconstruct a pattern classifier for each of the divided areas, based onan image-capturing range division method and recognition reliabilityinformation which are associated with the camera information. When thethree-dimensional recognition unit 6 performs three-dimensionalrecognition, in a case where a pattern identifier of thethree-dimensional information created from the sample data is used as athree-dimensional information template, the pattern identifierreconstruction unit 70 d may reconstruct a pattern identifier of thethree-dimensional information.

The recognition parameter determination unit 70 e changes parameters ofan algorithm by which the captured image recognition unit 5 performsimage recognition and the three-dimensional recognition unit 6 performsthree-dimensional recognition, based on a reconstructed patternidentifier, camera information, a captured image, a parallax image, animage-capturing range division method, and recognition reliability.

As a method by which the recognition parameter determination unit 70 echanges parameters of a recognition algorithm, there is a method ofchanging a parameter that determines a size or a shape of a detectionwindow based on camera information such as a depression angle of thecamera 2 and lens distortion.

In addition, as a method of changing the parameter, the recognitionparameter determination unit 70 e first performs scene recognition andthe like, based on the captured image and the parallax image, anddetermines a position where an object constantly having movement of animage such as an escalator exists within an image-capturing range, aplace where a sunlight change is large, and the like. Then, there is amethod in which the recognition parameter determination unit 70 eadjusts a parameter used when a matching degree between a local featurepart in an image and an image feature of a pattern identifier iscalculated, by the pattern identifier such that erroneous detection isreduced.

As an example in which the recognition parameter determination unit 70 echanges a parameter used for three-dimensional recognition, there is amethod of changing a block size of stereo matching in a case whereparallax that can be calculated from camera information or a parallaximage is determined to be unstable. In addition, there is a method ofadjusting an extraction range when the three-dimensional recognitionunit 6 extracts three-dimensional information of the head from theparallax image. As a method of changing the parameter of the recognitionalgorithm, the parameter may be adjusted for each divided area, based onan image-capturing range division method associated with the camerainformation.

The recognition reliability updating unit 71 updates recognitionreliability matrix data acquired by the area information acquisitionunit 70 b, based on the information of the pattern identifierreconstructed by the recognition function adjustment unit 70 and theinformation of the changed parameter of the recognition algorithm.

As a method of updating the matrix data, there is a method of improvinga value of the recognition reliability of a corresponding objectrecognition function in a case where a parameter of the patternidentifier or the recognition algorithm is changed in the entireimage-capturing range. In addition, there is also a method or the likein which, in a case where only the pattern identifier and the parameterassociated with an area where an image-capturing range is divided by thedivision method acquired by the area information acquisition unit 70 bare changed, a value of recognition reliability corresponding to thearea is improved.

The recognition reliability calculation unit 7A includes a camerainformation acquisition unit 7 b, an area information acquisition unit 7c, and a recognition reliability determination unit 7 d. The recognitionreliability determination unit 7 d receives camera information acquiredby the camera information acquisition unit 7 b, an image-capturing rangedivision method and the recognition reliability matrix data acquired bythe area information acquisition unit 7 c, and the recognitionreliability matrix data updated by the recognition reliability updatingunit 71. Then, the recognition reliability determination unit 7 ddivides the image-capturing range into one area or a plurality of areas,determines recognition reliability for each area, outputs therecognition reliability for each determined area to the recognitionresult combining unit 8.

According to the object recognition device 1A described above, a patternidentifier and a parameter of a recognition algorithm are changed forthe entire image-capturing range or for each divided area, and a valueof the corresponding recognition reliability is updated. Thereby, it ispossible to detect an object in a captured image or a parallax imagewith high accuracy, regardless of an installation environment of thecamera 2 or a position of an object.

[3. Third Embodiment]

Next, an object recognition system according to a third embodiment ofthe present invention will be described.

FIG. 14 illustrates an internal configuration example of an objectrecognition system 10B according to the third embodiment.

The object recognition system 10B includes one camera 2, a distancemeasurement device 13, an object recognition device 1B, a database 9, adisplay unit 11, and an operation unit 12.

The object recognition device 1B includes a distance informationacquisition unit 14, a distance image generation unit 15, and a distanceimage recognition unit 16, in addition to the one image acquisition unit3, the captured image recognition unit 5, the recognition reliabilitycalculation unit 7, and the recognition result combining unit 8. Theobject recognition device 1B recognizes an object, based on a capturedimage and a distance image in which distance information to an object isembedded in the captured image.

The distance measurement device 13 (an example of a measurement device)measures a distance to an object in an image-capturing range captured bythe camera 2. For example, a distance sensor, a laser radar, an infraredtag, or the like is used as the distance measurement device 13.

The distance information acquisition unit 14 acquires distanceinformation for each object measured by the distance measurement device13.

The distance image generation unit 15 generates a distance image inwhich the distance information acquired by the distance informationacquisition unit 14 is embedded in the captured image acquired by theimage acquisition unit 3.

The distance image recognition unit 16 (an example of a recognitionunit) recognizes an object, based on the distance information includedin the distance image. At this time, the distance image recognition unit16 recognizes the object by a measurement range division method andrecognition reliability matrix data which correspond to the distancemeasurement device 13, and outputs recognition results to therecognition result combining unit 8. The measurement range divisionmethod and the recognition reliability matrix data which correspond tothe distance measurement device 13 are read from the database 9 by thedistance image recognition unit 16. Then, the distance image recognitionunit 16 calculates a matching degree between an object in an overheadview image obtained by converting a viewpoint of the distance image anda template read from the database 9.

The recognition reliability calculation unit 7 calculates recognitionreliability of a function of recognizing an object within a capturedimage by the captured image recognition unit 5, and recognitionreliability of a function of recognizing an object within a distanceimage by the distance image recognition unit 16.

The recognition result combining unit 8 combines a matching degreeindicating that an object recognized by the captured image recognitionunit 5 from a captured image is a specified object and recognitionreliability for recognizing an object in the captured image. Inaddition, the recognition result combining unit 8 combines a matchingdegree indicating that an object recognized by the distance imagerecognition unit 16 from a distance image is a specified object andrecognition reliability for recognizing an object in the distance image.Then, in a case where a recognition sum value calculated by combiningrecognition results of the image recognition and recognition results ofthe distance image recognition, based on the recognition reliability isequal to or larger than a predetermined value, the recognition resultcombining unit 8 sets the specified object as detection results whichare detected.

According to the object recognition device 1B described above,recognition reliability of a captured image, recognition results of thecaptured image, recognition reliability of a distance image acquiredfrom the distance measurement device 13 and generated, recognitionresults of the distance image are combined, and the detection result ofan object is displayed on the display unit 11. In this way, even in anenvironment where only one camera 2 is installed, detection accuracy ofan object can be improved by using the distance measurement device 13 incombination.

[4. Modified Example]

In each embodiment described above, the object recognition devices 1,1A, and 1B may include the database 9. In addition, the objectrecognition devices 1, 1A, and 1B may include the display unit 11 andthe operation unit 12.

In addition, the object recognition system according to each embodimentdescribed above can be used as a human analysis system which analyzes ahuman in a specified place (for example, within a station building,within a building, a road), and can be used as a monitoring system. Atthis time, by using an infrared camera capable of capturing an image ofan object by infrared light as the camera 2, it is possible to recognizean object even at night and detect a specified object.

It is needless to say that the present invention is not limited to theabove-described embodiments, and various other application examples andmodification examples can be obtained as long as the applicationexamples and the modification examples do not deviate from the gist ofthe present invention described in the claims.

For example, in order to describe the present invention in aneasy-to-understand manner, the above-described embodiments describeconfigurations of a device and a system in detail, and are not limitedto those including all the configurations described above. In addition,it is possible to replace a part of the configuration of a certainembodiment with a configuration of another embodiment, and furthermore,it is also possible to add a configuration of another embodiment to theconfiguration of the certain embodiment. In addition, it is alsopossible to add, delete, and replace other configurations with respectto part of the configuration of each embodiment.

In addition, control lines and information lines indicate what isconsidered to be necessary for explanation, and it is not limited thatall the control lines and the information lines are necessarilyillustrated on products. In practice, it may be considered that almostall the configurations are connected to each other.

REFERENCE SIGNS LIST

1: object recognition device

2: camera

3: image acquisition unit

4: distance calculation unit

5: captured image recognition unit

6: three-dimensional recognition unit

7: recognition reliability calculation unit

8: recognition result combining unit

9: database

10: object recognition system

11: display unit

12: operation unit

The invention claimed is:
 1. An object recognition device comprising: aprocessor configured to: recognize an object by a plurality offunctions, based on information which is obtained by measuring theobject by a plurality of measurement devices; calculate recognitionreliability of recognition results for each function; and combinerecognition reliability of the object and the recognition results,detect a specified object, and output detection results of the specifiedobject, wherein in a case where information that is obtained bymeasuring the object by the plurality of measurement devices is acaptured image which is obtained by capturing an image of the object anda parallax image which includes distance information to the object, theobject is recognized based on the captured image, and the object isrecognized based on the distance information which is included in theparallax image, a matching degree is calculated between a patternidentifier that is generated based on sample data which is read from adatabase and a feature part of the object that is extracted from thecaptured image, a matching degree is calculated between a template thatis read from the database and three-dimensional information of theobject in an overhead view image that is obtained by converting aviewpoint of the parallax image, and recognition reliability iscalculated for recognizing the object in the captured image andrecognition reliability is calculated for recognizing the object in theparallax image.
 2. The object recognition device according to claim 1,wherein the processor is further configured to: acquire the capturedimage and the parallax image from the measurement device, acquiremeasurement device information of the plurality of measurement devicesfrom the database, acquire an image-capturing range division method andmatrix data of the recognition reliability which correspond to themeasurement device information from the database for each area, anddetermine recognition reliability for recognizing the object in thecaptured image and recognition reliability for recognizing the object inthe parallax image for each of the areas which are obtained by dividingthe image-capturing range of the captured image in predeterminedposition and size, based on the image-capturing range division method.3. The object recognition device according to claim 2, wherein in a casewhere results which are obtained by combining a degree indicating thatthe object which is recognized from the captured image is the specifiedobject, a recognition reliability for recognizing the object in thecaptured image, a degree indicating that the object which is recognizedfrom the parallax image is the specified object, and recognitionreliability for recognizing the object in the parallax image, are equalto or larger than a predetermined value, the obtained results are set todetection results of the specified object.
 4. The object recognitiondevice according to claim 3, wherein the measurement device information,the division method, and the recognition reliability can be changed byan input from an operation unit through a graphical user interface (GUI)which is displayed on a display unit, and the measurement deviceinformation, the division method, and the recognition reliability whichare changed are recorded in the database, and wherein the recognitionreliability is calculated for each of the areas, based on themeasurement device information, the division method, and the recognitionreliability which are changed and acquired from the database.
 5. Theobject recognition device according to claim 4, wherein the processor isfurther configured to: adjust a function of recognizing the object inthe captured image and a function of recognizing the object in theparallax image, based on the measurement device information; and updatethe matrix data in which the recognition reliability is stored, based ona function of recognizing the adjusted object.
 6. The object recognitiondevice according to claim 5, wherein the measurement device informationincludes at least one of an installation environment of each measurementdevice and internal information of each measurement device.
 7. An objectrecognition device comprising: a processor configured to: recognize anobject by a plurality of functions, based on information which isobtained by measuring the object by a plurality of measurement devices;calculate recognition reliability of recognition results for eachfunction; and combine recognition reliability of the object and therecognition results, detect a specified object, and output detectionresults of the specified object, wherein in a case where informationthat is measured by the plurality of measurement devices is a capturedimage which is obtained by capturing an image of the object and adistance image in which distance information to the object is embeddedin the captured image, the object is recognized based on the capturedimage, and the object is recognized based on the distance informationwhich is included in the distance image, a matching degree of the objectis calculated between a feature part that is extracted from the capturedimage and a pattern identifier that is generated based on sample datawhich is read from a database, a matching degree is calculated betweenthe object in an overhead view image that is obtained by converting aviewpoint of the distance image and a template that is read from thedatabase, and recognition reliability is calculated for recognizing theobject in the captured image and recognition reliability is calculatedfor recognizing the object in the distance image.
 8. An objectrecognition system comprising: a plurality of measurement devices thatmeasure an object; a recognition unit that recognizes an object by aplurality of functions, based on information which is obtained bymeasuring the object by the plurality of measurement devices; arecognition reliability calculation unit that calculates recognitionreliability of recognition results that are obtained by recognizing theobject by the recognition unit, for each function; a combining unit thatcombines recognition reliability of the object and the recognitionresults, detects a specified object, and outputs detection results ofthe specified object; and a display unit that displays the detectionresults, wherein in a case where information that is obtained bymeasuring the object by the plurality of measurement devices is acaptured image which is obtained by capturing an image of the object anda parallax image which includes distance information to the object, theobject is recognized based on the captured image, and the object isrecognized based on the distance information which is included in theparallax image, a matching degree is calculated between a patternidentifier that is generated based on sample data which is read from adatabase and a feature part of the object that is extracted from thecaptured image, a matching degree is calculated between a template thatis read from the database and three-dimensional information of theobject in an overhead view image that is obtained by converting aviewpoint of the parallax image, and recognition reliability iscalculated for recognizing the object in the captured image andrecognition reliability is calculated for recognizing the object in theparallax image.